Evaluating Nearest Neighbours in Avalanche Forecasting - a Qualitative Approach to Assessing Information Content

Nearest neighbours (NN) approaches are a statistically-based pattern classification technique used extensively for computer-assisted decision making in tasks such as ski area and winter road management and backcountry avalanche forecasting. The essential hypothesis behind NN assumes that the available data (normally current meteorological and snowpack data) usefully select similar avalanche conditions from the past which are thus of interest to the forecaster. Most evaluations of NN rely on verification schemes based on avalanche events and delivering summary statistics of performance in the form of contingency tables. We argue that such approaches, whilst useful, oversimplify NN's information output, and present a complementary approach to verification. Over two winters qualitative information was reported by forecasters, describing meteorological conditions, snowpack conditions, snow stability and related information. These information were entered in a web log, where forecasters could quickly and easily enter their assessment of the current situation. Using these data we have reevaluated the quality of information delivered by NN, by comparing NN forecasts with randomly generated forecasts. Our results suggest that, although the performance of NN may appear good when measured by summary statistics, the usefulness of the information presented to the forecaster may often be low with the events selected by NN as 'examples' not corresponding to the general avalanche situation e.g. NN selects single, skier triggered avalanches though on the forecast day large natural avalanches occurred. These findings demonstrate that, firstly, our previous evaluation approaches insufficiently described the information content that we wished to evaluate and secondly, suggest weaknesses in the NN approach. While NN demonstrates considerable skill in selecting similar weather patterns, it has considerably less skill in selecting similar snowpack and stability patterns. Finally, forecasters reported that filling in web logs was itself a useful process and led to more self reflection in their decision making process.